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ABSTRACT 

The purpose of this study was to investigate the effectiveness of Emotional Motivational Feedback Message 
(EMFEM) in an online learning environment. This exploratory research was conducted using mixed method 
single case study design. Participants were 15 undergraduate students enrolled in an instructional technology 
course in a large state university located in an urban region in the southeastern part of the United States during 
Fall 2013. Multiple data collection strategies were employed in this study such as a course interest survey, IT 
attitudes survey, open-ended questionnaires, research journals, forum/discussion postings, emails, reflection 
papers and warm-up surveys. The findings showed EMFEM helped to increase and maintain students’ 
motivation. Students liked the EMFEM and the style of the instructor’s teaching by giving emotional 
motivational feedback. Students had a closer and friendlier relationship with the instructor; they also started to 
use more emotional content themselves. 

1. INTRODUCTION 

Feedback, an important technique for learning, has been recognized as responses to students’ behaviors, tasks, 
assignments, and outcomes. Sprenger (2005) defined feedback as assistance to students in helping them know 
where they are in the learning process. Feedback can affect students in many ways, including improving learning 
(Askew, 2000; Cheng, Liang& Tsai, 2015;Mory, 2004; Sprenger, 2005), motivation (Connellan, 2002), and 
emotion (Burke & Pieterick, 2010). In traditional (face to face) classrooms feedback is a rapid, fluid, and almost 
constant process of natural interaction between the instructor and students. But in online environments the 
communications channels for feedback are generally somewhat constrained, making it more important that 
feedback be approached deliberately and with an eye to maximizing the efficacy of the limited feedback 
opportunities. 

Emotions are important factors in learning because they affect students’ success (Burke & Pieterick, 2010; 
Meyer & Turner, 2006), motivation (Flannula, 2006) and satisfaction (Cho & Fleron, 2015). Emotions also give 
clues to educators about what is happening in classroom environments during instruction (Meyer & Turner, 
2006). 

Connellan (2002) identified three different types of feedback: motivational, informational, and developmental. 
The different types of feedback help students remain engaged in the learning process (Askew, 2000), but 
motivational feedback messages are one of the most powerful types of feedback (Connellan, 2002). Many 
researchers have investigated how feedback messages improve student motivation levels and keep them highly 
motivated (e.g., Borham-Puyal & Olmos-Miguelanez, 2011; Brookhart, 2008; Kim & Keller, 2008; Nicol & 
MacFarlane-Dick, 2006; Van den Bossche, Segers, & Jansen, 2010). Motivational feedback messages can help 
students during the learning process by encouraging them about their progress (Sprenger, 2005). The reactions of 
students to motivational feedback are generally emotional (Burke & Pieterick, 2010). These emotional reactions 
help students see feedback as a personal message, which makes it a powerful tool to keep them more motivated 
(Kim & Keller, 2008). 

In this study, we used an Emotional Motivational Feedback Message (EMFEM), which provides feedback by 
using motivational strategies and adding specific emotional content. EMFEM could be important to motivating 
students because it could affect students both extrinsically (as a reinforcement) and intrinsically (as an 
encouragement). Extrinsically, EMFEM provides reinforcements from the instructor to the student for doing 
better and/or improving their learning activities, leading to the student’s being motivated to continue in the same 
manner (Ryan &Deci, 2000). When the student internalizes those reinforcements, coming to understand that 
because they have learned successfully in the past, then they can learn successfully in future tasks, they become 
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“moved to act for the fim or challenge entailed rather than because of external prods, pressures, or rewards.” 
(Ryan & Deci, 2000, p.60). 

An Emotional Motivational Feedback Message is a feedback message which includes both motivational 
strategies and emotional content for motivating and encouraging students to seek to learn more about a specific 
topic. Kim and Keller (2008) investigated motivational feedback messages in online learning environments by 
using motivational components of the ARCS model (Keller, 1987), and adding volitional components such as 
Gollwitzer’s Rubicon model, (Gollwitzer, 1999), Kulh’s six action control strategies (Kulh, 1987), and Visser 
and Keller’s (1990) strategy of motivational messages, to design feedback email messages. This study was based 
on the ARCS model because that model has long been used for designing motivational messages (e.g., Kim & 
Keller, 2008; Visser & Keller, 1990). In addition, research shows that the ARCS model works efficiently for 
designing motivational feedback messages (ChanLin, 2009; Cheng & Yeh, 2009). In this study we used a few 
different and additional components to explore the effectiveness of EMFEM, which is grounded in feedback 
strategies (Brookhart, 2008), Visser and Keller’s (1990) motivational message design (which was influenced by 
Keller’s, 1987, ARCS theory), and emotional content strategies (Ekman, 2003; Goddard, 2011; Kappas & 
Kramer, 2011). 

Instructors must pay attention to emotion in online learning because in education, it helps to support students in 
achieving their goals (Meyer & Turner, 2007; Yunus, Osman, & Ishak, 2011). Emotions can be transferred in 
many ways, such as using facial expressions and mimicry (Ekman, 2003; Marinetti, Moore, Lucas, & Parkinson, 
2011; White & Gardner, 2012), gestures/postural movements (Marinetti et ah, 2011; White & Gardner, 2012), 
and verbal emphasis (Marinetti et ah, 2011; White & Gardner, 2012). Unfortunately, these techniques are not 
applicable in text-based asynchronous online learning. Text-based options to facilitate the transfer of emotions 
include considering the semantic value of words (Farrell, 2012), bold/colored/underlined typing (Dweck, 
Mangels, & Good, 2004), and emoticons (Kappas & Kramer, 2011; Sarsar, 2008; Sarsar & Kisla, 2013). Because 
online learning environments are still mostly text-based learning environments, ways of adding emotional 
content to online feedback messages are limited. Using the meaning of the words, formatting the words by using 
font effects, and adding emoticons to get students’ attention are some strategies that can infuse emotional content 
into messages. 

1.1. Semantic value of words. The meaning of words may affect students’ emotions (Anusha & Sandhya, 2015; 
Goddard, 2011). While giving feedback, each word has a specific meaning to the students. If a teacher writes 
“this is great work,” it might make students feel happy. Whereas "this is good work" might elicit a positive but 
less pronounced response. Ekman (2003) highlighted that words are the primary way of representing emotions in 
text-based feedback; therefore, the selection of words is important in generating an emotional response. 

1.2. Font effects. Using font effects is another option to make messages emotional. This is one of the true 
affordances of online learning environments. The majority of online environments use word-processing editors 
which allow both teachers and students to write their feedback using italic, bold, and/or colored text. Price, 
McElroy & Martin (2015) stressed that font effects might improve students’ recall performance. Kalra and 
Karahalios (2005) noted that different textual representations, such as different colors, font types, size, and 
format, can help to express emotional content. 

1.3. Using emoticons. Emoticons are another way of expressing emotions online (Dunlap, Bose, Lowenthal, 
York, Atkinson & Murtagh, 2015; Kalra & Karahalios, 2005; Kappas & Kreamer, 2011; Sarsar, 2008; Tossell, 
Kortum, Shepard, Barg-Walkow, Rahmati & Zhong, 2012). Flowever, commonly used emoticons have been 
more limited than the number of emotions in use. Tossell et al (2012) stated that the top three emotions are 
happy (“:)”), sad (“:(“), and very happy (“:D”). The other emoticons that have been commonly used are Joyful 
:p, Wink/Joking ;), Surprise :o. Anger :-[, and Uncertainty :-\ (Garrison, Remley, Thomas, & Wierszewski, 
2011 ). 

Emotion can be adaptable to all steps of the ARCS model, potentially making feedback more personal and more 
motivational. Using the three most common ways of expressing emotion in online environments might assist 
students because if the teacher would like to stress a point of view, he or she can make it bold, italic, in a 
different color, and/or use the emoticons to make message more emotional. 

There are many studies on feedback, motivation, and emotion in online learning environments; however, there is 
limited research on the effectiveness of motivational feedback email messages, and a review of the literature in 
preparation for this study revealed no research on effectiveness of emotional motivational feedback email 
messages. Visser and Keller (1990) designed a clinical use of motivational messages to help adult learners in an 
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in-service continuing education program. They modeled their study as an embedded single case study by using 
mixed method design to implement and test motivational message design of the ARCS model. They used a 
variety of data collection tools, such as weekly questionnaires, round-table discussions, observations, and grades. 
They had 15 adult participants and the results showed that motivational message design provided positive 
influence on students’ motivation to learn. They also found the retention rate and students’ performance 
improved. 

With a similar interest, Kim and Keller (2008) investigated the effectiveness of supportive information by using 
motivational and volitional email messages which were based on different theories and methods, such as Keller’s 
ARCS model, Kuhl’s action control theory (Kuhl, 1987), Gollwitzer’s Rubicon model of motivation and volition 
(Gollwitzer, 1999), and Visser and Keller’s strategy of motivational messages (Visser & Keller, 1990). 
Motivational and volitional email messages were sent to 30 students (Personal Message Group) with personal 
messages and to 71 students (Non-Personal Message Group) without personal messages. The results showed that 
the personal message group evinced a higher level of motivation, especially in regard to confidence, than the 
Non-Personal Message Group. Kim and Keller conducted another study to examine the effectiveness of 
motivational and volitional email messages on pre-service teachers’ motivation, volition, and performance, in 
addition to their attitudes toward technology integration (Kim & Keller, 2010). Fifty-six pre-service teachers 
participated in this study. The results signified that motivational and volitional email messages worked as an 
effective tool for facilitating pre-service teachers’ positive attitudes toward technology integration and supported 
a higher volition and more positive attitude toward technology integration. 

In another study, Sampasivam and Wang (2012) determined that their learners’ math test anxiety was associated 
with changes in their achievement goals and task-specific emotions in response to different types of feedback. 
Ninety-five students participated in their study, and were randomly assigned to a Negative Feedback Condition, 
Positive Feedback Condition, or Control condition. The results showed that there was no significant difference in 
interaction between feedback condition and math anxiety, and multivariate effects for both math anxiety and 
feedback condition, but feedback had a significant effect on participants who received positive feedback, while 
students in the negative feedback condition reported the most negative effect. 

Terzis, Moridis, and Economides (2012) explored the effect of emotional feedback on behavioral intention to use 
computer based assessments (CBA). They used a virtual agent for giving emotional feedback. The agent 
appeared female, and was rendered in 3D to express common facial emotions, such as sadness or fear. They 
administered a survey questionnaire to 134 students. They observed that emotional feedback had a direct effect 
on behavioral intention to use a CBA system and on other crucial determinants of behavioral intention. This 
study highlighted that emotional feedback makes computer-based assessments seem playful and easy to use. 

Researchers (e.g., Maier, Wolf & Randier, 2016;Pintrich & Schunk, 2002; Schutz & Pekrun, 2007) have shown 
that emotions in education affect students’ achievement and motivation. Flowever, there is limited research on 
emotions in online education, especially in text-based asynchronous online learning environments, and this study 
contributes toward closing that gap in the literature. 

The purpose of this study was to investigate the effectiveness of EMFEM in online learning environments. This 
exploratory research employed a mixed method, single case study design. Visser and Keller’s (1990) 
motivational message design, which was influenced by Keller’s (2010) ARCS Motivational Design Model, was 
selected as the theoretical framework for this study. This study was designed to investigate the effectiveness of 
EMFEM on students' motivation, and attitudes in online learning environments with the intent of addressing the 
following research question: How effective are emotional motivational feedback messages in an online learning 
environment? 

2. METHODOLOGY 

To investigate the effectiveness of emotional motivational feedback messages in an online learning environment, 
we collected data within a case study of an undergraduate online course taught at a large university in the 
southeastern United States. The course provides training in contemporary computer skills, including topics such 
as information literacy, cyber ethics, spreadsheets, and web design. The course included 20 students, 15 of whom 
agreed to participate in the study after signing informed consent documents. 

2.1. Participants 

The 15 participating students were in two academic areas: 11 were majoring in Exercise Science (which required 
this course as part of the program of study), and 4 were majoring in fields in the Department of Communication 
(which allowed the course to count as an elective). Ten of the students were male, and 5 were female. At the 
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beginning of the course, participants completed Keller’s (2010) Course Interest Survey, which consists of 34 
items identifying 4 subscales of interest (attention, relevance, confidence, satisfaction). Possible total scores can 
range from 34 to 170. In this sample, the minimum score was 102, and the maximum score was 158, with a mean 
of 133.40 (SD = 18.44). The mean for female students, 139.40 (SD = 10.00) was greater than the mean for male 
students, 130.40 (SD = 21.32), but the difference was not statistically significant (t(13) = .884, p> .10) within 
this small sample. The mean for Exercise Science students, 135.18 (SD = 19.15) was greater than the mean for 
Communication students, 128.50 (SD = 17.92), but this difference also failed to evince statistical significance 
(t(13) = -.606,p> . 10). 

2.2. Setting 

The online computer skills class that provided the setting for this study was a 14-week, 3 credit hour course 
offered through the University’s online learning management system (LMS). Students registered for the course 
during the University’s regular registration period, just as they would for a traditional, face-to-face class. The 
lead author served as the instructor for this course. Over 13 weeks, students submitted weekly assignments, and 
they submitted reflection papers during the 14th week, giving ample opportunity for the instructor to provide 
emotional motivational feedback. Feedback on assignments was delivered through the “Dropbox” area of the 
course site, and it consisted of a numeric grade along with written feedback. Additionally, students 
communicated with each other and with the instructor through emails, the feedback box, and discussion/forum 
postings. During this section of the course, the instructor published approximately 150 emails, 250 forum 
postings, and 300 feedback messages, although only those involving study participants are considered in this 
report. 


2.3. Procedures 

At the beginning of the course, students completed the Course Interest Survey (Keller, 2010) and the Information 
Technology Survey (Wong, 2002), which measures participants’ attitudes toward information technology. The 
survey consists of 24 Likert-type items with possible scores ranging from 1 {strongly disagree) to 5 {disagree). 
Additionally, we administered a questionnaire of 10 open-ended items designed to further investigate students’ 
motivational factors and attitudes toward information technology and the course. 

Also at the beginning of the course, we administered three mini-surveys to determine how familiar students were 
with using emotional content strategies. These surveys asked about color preferences, typing/font style 
preferences, emoticons, and encouraging and discouraging words. Results from these surveys guided the 
instructor (first author) in his creation of emotional motivational feedback messages throughout the course. 

Each week of the course, students engaged in the instruction provided and then responded through their 
assignments, discussion board postings, and emails. After the instructor evaluated their contributions each week, 
he responded to them with emotional motivational feedback messages, which he had created using the mini¬ 
survey results, Keller’s (2010) ARCS strategy, feedback strategies (Brookhart, 2008), and emotional content. 
Brookhart (2008) identified four strategies—timing, amount, mode, audience—that were used throughout the 
course. The feedback messages were given after each assignment and on a weekly basis. They covered main 
points and level of achievement of major course objectives. The instructor sent feedback messages individually 
using email-messaging or other relevant tools. 


Table 1 .Constructing Emotional Motivational Feedback Messages Using Keller’s (2010) ARCS Model 


Stage Description 


Key Concepts 


Example Text 

Emotional Motivational Feedback Messages 


G 

O 


V 

< 


Capturing 

interest 


Write student’s name at 
beginning of message 
Use humor 


John. 

Thank you for sending this GREAT home¬ 
work. © 


u 

o 


G 

ca 

> 

u 




Meeting the 
personal needs 
and expectations 
for positive 
attitude 

between activity and 
student’s field. 


This homework was about the ways of using 
MS office Word efficiently. All of us using 
one of word editors to save our documents 
electronically. I am also one of them. © You 
are majoring in health care, so it was a very 
good opportunity for you to learn how to use 
this software for reporting the health 
information of your patient! 


• Identify goal of previous 
assignment 

• Describe relationship 

between goal and activity 

• Describe relationship 
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d 

o 

a 

D 


■o 

IS 

s 

o 


U 


o 

o 


Kj 

CO 


c 


Awareness of • 
achieving and 

controlling • 

success 

• 


Reinforcement • 


Confirm meeting of You met all the expectation of this small 

expectations activity. YAY! We used MS Word, but I 

Give advice regarding recommend you to try Open office as well, 
improvement of skills You will see that you can do the same kind of 

Provide information about tasks using other word editors, 
quality and correction (if 

needed) _ 

Provide grade for This excellent homework got 6 full credits!!! 

assignment YOU did a GREAT job. I know that you will 

Praise student effort do the same for others. © 


Keller’s (2010) motivational strategies were reflected mostly in the structure of content, which was focused, 
clear, personalized, and brief. Because the students received feedback as 

they were beginning the next week’s instruction, the instructor used motivational strategies to encourage their 
learning activities. Table 1 shows the phases of construction of a motivational feedback message and an 
emotional motivational feedback message using Keller’s ARCS model. Additionally, we felt it important to 
provide personalized but brief feedback, as longer feedback messages risk becoming tedious and confusing. 

Once the motivational feedback messages were created, the instructor introduced additional emotional 
components using three strategies: selecting words for their positive connotations, using font effects, and adding 
emoticons. The students had identified words with positive connotations in the mini-surveys, and the instructor 
made use of these. Font effects included using boldface, italics, underlining, colored text, colored backgrounds 
behind texts, or punctuation. Emoticons are computer images created using standard keyboard keys. For 
instance, :) represents a smiling face (©) when viewed on its side. Emoticons are popular in online forums and 
in text-messaging. 

2.4. Data Collection and Analysis 

Quantitative data were collected primarily through two instruments, Keller’s (2010) Course Interest Survey and 
Wong’s (2002) Information Technology Survey. Qualitative data were collected through the various 
assignments, discussion postings, and email messages previously described. Additionally, qualitative data were 
collected through open-ended questionnaires, the instructor’s electronic research journal, the mini-surveys, and 
end-of-course reflection papers. 

Because our sample size was small, we analyzed the quantitative data from the surveys using a descriptive 
analysis approach, identifying differences between subgroups of the participants, while understanding that these 
differences would not be generalizable to larger populations because of our purposive sampling strategy. Results 
of the quantitative analysis contribute to a detailed description of the research setting and participants, 
information readers can use to assess the transferability of the results. 

Because of the large number of text documents providing qualitative data for the study, we used content analysis 
to analyze them. As the main focus of the study was a new topic in online learning environments, content 
analysis was appropriate because it can lead to the development of a framework to categorize codes that then 
help conceptualize the content of the texts (Ulrike, 2010). We used Schreier’s (2012) description of the steps for 
content analysis to guide recursive analysis. We began by looking at the raw data for theme codes. When we had 
accumulated a number of codes, we used them to create a code sheet, and we sorted the data into categories. We 
used the code sheet to analyze the next group of data and then compared these results with previous results in the 
data collection. Once we created a draft code frame and categories, we sent the code frame and a sample of data 
to two colleagues for their review. Finally, the results from the peer reviewers were compared and the code 
frame was finalized. After data coding finished, we abstracted the data. 

Our data collecting involved multiple data sources (multiple participants who produced multiple types of texts), 
creating a triangulation (Krefting, 1991; Merriam, 1998) of results to contribute to the credibility of the study. 
We relied on peer review, as described above and in debriefing sessions as we conducted the study, to strengthen 
the dependability of the study. The instructor kept detailed notes in his research journal and during coding 
sessions in order to create an audit trail describing the research process as it occurred. These characteristics along 
with the descriptions we provide for the purposes of transferability evaluation support the trustworthiness of our 
results. 
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3. FINDINGS 

In this section, we present the baseline data we collected using the mini-surveys, and then we present the 
outcomes of the study group according to specific themes that appeared most prominently in our content analysis 
of the course texts. The baseline data presented here addresses the three primary strategies for including 
emotional motivational feedback messages in an online course, providing information about the participants’ 
experiences and attitudes toward the various feedback techniques. The outcomes are presented in the following 
categories: motivation, attitudes, instructor’s role, emotional reflection, and feedback. 

3.1. Baseline Data 

The three mini-surveys conducted at the beginning of the course provided an understanding of students’ previous 
experience with the emotional motivational feedback strategies we studied: word choices, font effects, and 
emoticons. These data provide a description of the students as they began the course and provided guidance to 
the instructor as he designed emotional motivational feedback messages during the course. 

3.1.1. Word Choices. Given a choice of potential feedback for an assignment, most student participants selected 
the choice they characterized as motivating (“This assignment should be improved, but I believe that you could 
do better.”) over the choice they characterized as honest (“This assignment is not good enough, but I believe that 
you could do better.”). One participant preferred what we intended as a negative phrasing (“This assignment is 
bad, but I believe that you could do better.”) but only explained this preference as a “personal choice.” 

Participants identified a number of words as positive, as making them feel happy when they heard them. The five 
most frequently appearing words were “excellent,” “wonderful,” “great,” “beautiful,” and “perfect.” Similarly, 
we asked them to identify negative words, words that might make them feel upset when they heard them from 
somebody, and the five most frequently occurring of these were “bad,” “terrible,” “horrible,” “awful,” and 
“stupid.” 

3.1.2. Font Effects. In the mini-surveys, participants mentioned that colored text, italics, and boldface can 
emphasize the importance of the meaning of texts. However, students differed on how they interpreted that 
importance, particularly with regard to colored text. On the mini-survey, colored text was represented as an 
option using red, which, for one student, conveyed a positive emphasis “because it is a different color, which 
stands out the most among the other fonts.” However, another student felt it a negative influence “because the 
red throws me off a bit. Red is usually associated with something bad/negative.” His opinion was shared by 
another participant, who also discussed other font effects from the mini-survey: 

1 like GREAT [the sample word] bolded and capitalized. It affects me positively and makes me 
feel like I did a really good job. Black is always a safe professional color to use especially for 
bold. 1 like colors bolded as well, and I wouldn't mind them used, but not in red. I associate red 
with correction and wrong doing so seeing GREAT in red creates a confused reaction for me. 

How students felt about using colored fonts for emphasis was confounded by the fact that a number of students 
had negative associations with the particular color being discussed (red). The participants might have had more 
positive things to say about a different font color, such as blue or green, although some participants might have 
had negative associations with one or both of those colors as well. 

The participants suggested a number of ways they might themselves use font effects in feedback they provide. 
One wrote, “I would use them because they would be a great way to highlight the key points in the feedback.” 
Another viewed the font effects as useful as a back-up plan: “I may use it if the student seems to not notice the 
first few feedbacks about what I’m trying to hint towards.” A third student emphasized how font effects might be 
particularly relevant in online learning environments: “It is exciting and since you do not have face to face time 
together it is nice to give variety to students.” And one student described a specific scheme for using different 
font effects in feedback: 

I would use it on a word that could sum up the whole point. That way if students don't take the time to 
read the feedback, they will at least see the one word and know what the instructor felt about their 
assignment 

3.1.3. Emoticons. Most participants reported that they had used emoticons, with 6 of them reporting that they 
use emoticons sometimes and 7 reporting that they use emoticons often. However, 8 of the 15 participants 
indicated that they did not feel emotionally connected when they used emoticons in learning environments. For 
instance, one wrote, 
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I feel weird using these with professors since I use emoticons in an unprofessional friendly 
setting. School is like a job for me that is professional. 1 like using emoticons, but in class I 
feel like I have trained to never use things like that. However, I feel more connected when 
using them in everyday life, but in class or work it's just a different setting. 

For this student, using emoticons conflicted with the expectations he had developed regarding the professional 
nature of collegiate learning. Another student identified a similar restraint: “Sometimes, it just feels weird,” she 
said, “because teachers have told us to refrain from it for so long. But when I get more used to it. I'll probably be 
more emotionally connected.” 

In contrast, a third student identified a reluctance to use emoticons based on the medium itself: “I'm more 
focused on what and how an individual delivers a message. Emoticons are nice for emphasis and accentuating a 
message, but some people just like to use it for the sake of using it.” She suggests meanings conveyed by 
emoticons may be diminished by their overuse by some communicators. 

3.2. Outcomes 

3.2.1. Motivation. At the end of the course, the participants again completed the Course Interest Survey, and the 
results are presented in Table 2. On all four of the ARCS (Keller, 2010) subscales, mean scores were higher than 
the subscales midpoint, suggesting a high level of motivation among the participants. More evidence of this 
effect on motivation appeared in students’ responses to the post-intervention questionnaire. “As a college 
student,” one student wrote, “stress hits you from so many different angles and that one feedback message could 
give you that extra push,” suggesting that the feedback messages contributed to his confidence. Another student 
explained, “encouragement, emotional feedback, is definitely something that allows students to push further in 
their studies and connect with a professor in a way that makes class enjoyable,” suggesting that the feedback 
messages contributed to her confidence and satisfaction. In her reflection paper, one of the students also 
described how the feedback messages contributed to her confidence: “It shows that the instructor is staying 
positive and trying to encourage you to do better.” 


Table 2. CIS Survey with Subcategories 


Subscale 

Mean Score 

Scale Middle Score 

Scale Maximum Score 

Out of 5 

Attention 

29 

24 

40 

3.60 

Relevance 

36 

27 

45 

4.00 

Confidence 

32 

24 

40 

4.00 

Satisfaction 

36 

27 

45 

4.00 

Total 

133 

102 

170 

3.99 


3.2.2. Attitudes. Because this study was conducted within an instructional technology course, we surveyed the 
participants about the course to see if they had positive feelings about it. The Instructional Technology Attitude 
survey produced a mean score of 87.80, which is just above the midpoint of the scale, suggesting a slightly 
positive mean for the group of participants. In their reflection papers, a number of students described their 
attitudes about the course. One wrote, “I appreciated everything this class was, especially compared to my other 
classes,” and another wrote, “I feel like the course was exactly how I imagined it would be. I learned good 
quality material that was very helpful and useful as I continued through the course.” A third wrote, 

Overall this was a great class to take and one that I would definitely recommend to everyone 
because it is not your average college class where you sometimes wonder if you are ever 
actually going to use the things that you are learning in life. 

While these statements do convey a positive assessment of the participants’ experiences in the course, they do 
not specifically tie that evaluation to the innovative use of feedback in the course. However, one student did 
make that connection in his reflection paper: 

Although I am not happy with my potential final grade for the class, I was more than satisfied with the 
professor’s attempt at helping me get a better grade throughout the semester. 

3.2.3. Instructor’s Role. Students’ descriptions of their beliefs about the course instructor and the relationship 
they had with him were evinced in their comments about the emotional motivational feedback messages. For 
instance, on the post-intervention questionnaire, one student wrote, “I liked it just because it made our 
relationship more [than] a instructor and student more like a friend level.” Another wrote, “They [feedback 
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messages] made me feel like I could contact my instructor whenever, which I did not feel with my other online 
instructors, because a close connection between a student an instructor is helpful for the student to want to 
learn.” In his reflection paper, one student described the instructor: “Gave great feedback, responded almost 
immediately to emails and made you feel that you weren’t just a student and an instructor but a friend who is 
helping you to become good at using the computer.” Another wrote, 

As the semester progressed I noticed the feedback of [instructor] becoming more personalized, 
as if he was genuinely concerned with something that I was doing, or the way I was absorbing 
the material, and I appreciate his individualized attention. 

A third student wrote, “This particular course was very interesting for me because I never had an instructor who 
expresses his feelings and emotions to his students.” These comments suggest that instructor’s efforts to convey 
emotional and motivational content within the feedback messages were recognized and appreciated by the 
students. 

As researchers, we were pleased at this kind of response from the students, but, in his research journal, the 
instructor described some of the difficulties he encountered while trying to provide detailed emotional 
motivational feedback. During Week 2 of the course, he wrote, 

I am already overwhelmed to write feedback, because I am writing very individually, so I 
needed to mention very specific points. Writing feedback to one student takes 20 minutes. I 
think it would be a great to create a new [template] and leave some parts empty for writing 
individual points. 

While creating the new template did help the instructor provide feedback to the students more quickly, it led to 
another complication. During Week 6 of the course, he wrote, 

Students’ expectations are increasing; they email me more and want me to respond very 
quickly. If I wait one day to respond, they are emailing me again. It is the 6 th week of the 
semester and it is getting more difficult to respond to their emails, because they started to write 
me more. It is the first time that I am dealing with that many emails. 

However, by Week 10 of the course, he himself began to feel emotional and motivational benefits from the 
frequent communication with his students: 

It is the 10 th week and I feel that I have become friends with them. They started to share their 
daily life with me. Although they haven’t seen me, they are sharing important events of their 
lives with me. I feel more connected while they are sharing more. It is the first time I feel that I 
know my students better. 

For the students, the emotional motivational feedback messages contributed to creating a bond between them and 
their online instructor, one they may not otherwise have had without face-to-face interaction with him. However, 
for the instructor, creating this relatable persona through the feedback messages took a great deal of time and 
effort, but ultimately he found that using the messages helped him relate better to his online students. 

3.2.4. Emotional Reaction. Students participating in the study reflected their feelings in different kinds of 
course texts, such as email messages, reflection papers, and discussion posts. Of the specific strategies for 
making texts more emotional that we are considering in this study, students mostly used punctuation, particularly 
exclamation points. Table 3 shows some of the strategies they used. Absent from the table (because students did 
not use them) are boldface, underlining, and colored text. 


Table 3.Strategies for Expressing Emotion 


Expression 

Frequency 

Sample 

Marks 

84 

I maybe we will have class together, good luck and great blog! 

I hope you have tons of fun in Texas!! 

Caps 

24 

REALLY glad football has started back up!!! 

I LOVE them! 

Emoticons 

21 

very cool :) 

The blog.:/ 

Internet Language 

13 

Lol! Exactly, I feel you man 

Finally get to try something I never got to do before haha. 

Value of Word 

12 

chiiiiiillllllllllll 

PS4 babyyyyyyyyyy!!!!!! 
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Students expressed their emotions about a number of course-related and non-course-related topics. For instance, 
one student emailed the instructor, saying, “I am very grateful that you were able to [make me] aware of this.” 
Another indicated, “1 love that the directions are clear and concise so I know how to format every assignment.” 
Another student, in a discussion post, describes in detail what he was feeling and why: 

This week has been one of my easiest weeks, [except] for the large test I had to study for on 
Monday, but other than that I had very few hours at work and a small amount of homework. I 
have nothing to complain about for the week except that it was boring. But everyone needs 
some boring sometimes 

One of the students commented on her fondness for the emotional motivational feedback messages: 

1 loved that there was feedback after most assignments because it gave me more of an 
understanding of what was expected and how well I was doing. I also liked that there was 
frequent communication from the instructor. 

This sharing of emotions evolved over time, according to the instructor’s research journal. During the first week, 
he observed that the students were “not ready to talk about their emotions,” but by Week 4, he realized, “When 
students see that I am comfortable [using] emoticons and typing formats, they started to use them as well. I think 
they were waiting for my move.” By Week 8, he had observed transformations in the students’ communication 
patterns: 

I think I started to change something in their lives. At least they feel free to talk about their 
daily lives’ issues with me. They don’t know me, I am just an online instructor for them but I 
also feel that we are getting closer and started to know each other better. I know some of their 
struggles and their problems. It makes me feel like more of a teacher. 

Thus, the emotional opening up of the students led to similar feelings within the instructor, who wrote in Week 
9, “Isn’t it good [when] one of your students [compliments] how you are taking care of your students? Yes it is. 
It made my day and I feel that I am connected with my students.” 

3.2.5. Feedback. Students found the feedback message important as a course component, one that was 
innovative compared to other courses they had taken. For example, in her reflection paper, one student wrote, 
This course also introduced me to emotional feedback. I noticed that my instructor’s 
feedback style was very different from my other instructors’ feedback. His style was way more 
personal and unique. 

Another student, also in her reflection paper, wrote, “This encouragement and feedback also sparked motivation 
to keep doing well on my assignments.” One student appreciated that he was able to provide feedback to the 
instructor throughout the course and that he received the same: 

Normally I give my feedback at the end the semester but this class wants the students to give 
feedback during the semester which is good because then the class itself can [make] 
improvements throughout the semester. One thing that is also good is the constant feedback 
every time I do an assignment. 

Students recognized the feedback messages as important components of the course, as opposed to evaluative 
statements, such as grades, to which they coidd not respond. “The instructor did send friendly and personalized 
feedback which helped,” one student wrote in his post-intervention questionnaire. 

A particular aspect of the feedback messages that students appreciated was personalization. Even when he was 
using a template to fashion messages, the instructor addressed students by name and included information 
specific to their assignments and backgrounds. In the post-intervention questionnaire, one student wrote, “it is an 
encouragement to read a personalized feedback,” while another observed, “I would not expect personalized 
feedback from other instructors.” One student wrote that the instructor’s “feedback made me feel like he actually 
took the time out to personally read our work and respond.” Several students expressed their appreciation of 
personalized feedback in their reflection papers. 

As previously mentioned, the instructor started to become overwhelmed with the amount of effort involved in 
providing feedback that reflected the ARCS (Keller, 2010) model, but he also recognized the benefits of putting 
in that extra effort. In his research journal, the instructor described some of the effects of feedback on the way 
students were performing. In Week 7, he wrote, 
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Writing a long feedback response is not a fun part of teaching but it is an essential part. It takes 
time, but I feel that my students are learning by reading feedback. Some of them are writing 
me back to ask if there is a chance to re-do their homework again. 

In Week 9, he recorded the following insight: 

Feedback is a very important tool for online learning, especially when your students cannot see 
you. They only know me by my feedback. I wonder what kind of instructor I am in their mind 
right now. I am very excited to read their reflection [papers]. 

In an educational institution, feedback, at least in the form of evaluative grades, is required. However, in this 
case, where the instructor was specifically infusing emotional and motivational content within the feedback he 
provided, the work-intensive process of providing feedback led to reflection of himself as a better, more 
involved instructor, one he hoped the students recognized as well 

4. DISCUSSION 

An emotional motivational feedback message (EMFEM) is defined in this study as a feedback message which 
was the combination of motivational strategies using ARCS model (Keller, 2010) and emotional content 
strategies: using emoticons (Kappas & Kreamer, 2011; Tossell et al 2012; Dunlap et al, 2016), formatting words 
(Kalra & Karahalios, 2005), and using the semantic value of words (Goddard, 2011; Anusha & Sandhya, 2015). 

The results of this study suggested that EMFEM contributed to an increase in students’ motivation. The Course 
Interest Survey (CIS) revealed that students were motivated during the online class. The CIS had four 
subcategories, attention, relevance, confidence, and satisfaction. They were measured individually in this study. 
Although all categories were higher than the midpoint of the survey, attention was slightly lower than the other 
three subcategories. Keller (2010) gives three main strategies to arouse and sustain attention. These strategies are 
perceptual arousal by using novel, surprising, incongruous, or uncertain events; inquiry arousal by having the 
learner generate questions, or a problem to solve; and variability by varying the elements of instruction. One of 
the strategies was to arouse attention in these kinds of environments to start writing messages by using the 
students’ names, as Kim and Keller (2008) highlighted in their studies. However, because of the nature of the 
course, assignments were designed weekly and feedback was mainly given after assignments. This meant that 
there was an entire week that students were engaging with their assignments. During a week, students’ attention 
might shift often. Additionally, students had more than three courses for the semester and some of them were 
taking an online course for the first time; therefore, their additional course load and lack of seeing the instructor 
might have affected their attention. Overall, students’ motivational level was increased. EMFEM was one of the 
catalysts for this increase during the study. These results are consistent with with Maier's et al. (2016) study. 
They found that elaborated feedback related to intrinsic motivation. 

EMFEM had two powerful components: Emotional and Motivational strategies. Emotion and motivational 
strategies might affect students’ motivation by using them individually and also, as shown in this study, by using 
them together. During the study, students mentioned that it seemed that EMFEM from their instructor motivated 
them in many different ways (i) EMFEM facilitated having a closer relationship with their instructor, (ii) 
EMFEM encouraged students to do their work better and (iii) EMFEM led to additional enjoyment in the course. 
The similar results reflected in Sampasivam and Wang's (2012) study mentioned that feedback had a significant 
effect on participants who had positive feedback, and students in the negative feedback condition reported the 
most negative effect. 

The result of the IT attitude survey showed that students’ attitudes toward the course improved by the end of the 
course. The other data also supported this statement. The CIS showed that satisfaction, as one of the 
subcategories of CIS, increased during the course. Students’ attitudes might be one of the indicators which was 
related to students’ satisfaction, as it was highlighted that e-learners’ satisfaction related to learners’ attitudes 
toward IT (Sun et al., 2008) or the course in this case. This finding also dovetails with the studies of Terzis et al. 
(2012) which highlighted that emotional feedback made computer- based assessments playful and easy to use. 
Similarly, Cho and Heron (2015) found that emotion played an important role in explaining students' 
satisfaction. 

Another aspect of positive attitude towards the course might be the effectiveness of EMFEM. Students’ 
reflection papers showed that students enjoyed having EMFEM during the course. They also reflected their 
positive attitude towards the course by mentioning how they liked to be in the class. This finding is consistent 
with Kim and Keller’s (2010) study, in which they highlight that motivational and volitional email messages can 
facilitate students’ positive attitude toward technology integration. 
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Personalization was one of the important effects of EMFEM. Gallien and Oomen-Early (2008) define 
personalized feedback as a feedback message which is given individually. This result might be explained under 
the title of feedback; however, it might be better to explain it separately for considering the limitation of 
asynchronous online learning environments and effectiveness of the feedback messages. There were several 
strategies applied in this area during this research. These include (i) mentioning students’ names in each 
EMFEM; (ii) mentioning specific information about them such as their field of study in the first EMFEM; (iii) 
giving specific points of their assignments in each EMFEM; and (iv) using emotional and motivational 
strategies. Students mentioned that EMFEM helped them feel personalized. On the other hand, personalized 
feedback messages might affect students’ satisfaction, as noted in Gallien and Oomen-Early’s (2008) study. 
They highlighted that students who received personalized feedback were more satisfied and performed 
academically better than students who received only collective feedback. 

One of the challenges of motivation in online classrooms is to make students feel that the instructor is addressing 
their individual needs. Lack of instructors’ personal attention might affect students’ motivation (Kim and Keller, 
2008). Students in this study noted that they would like to have personalized feedback messages in other classes 
as their instructor in this class had done. That way, they felt encouraged when they knew that the instructor gave 
his attention to all students’ work, read it and responded individually. As the instructor, the first author noted that 
personalized feedback was a part of the EMFEM, which made them feel like they belonged in the learning 
environment and encouraged students to do their work better. These results also overlap with Kim and Keller’s 
(2008) study. In their study, they highlight that personalized motivational volitional emails might be useful 
supports for improving students’ motivation. 

5. CONCLUSION 

This study focused on determining the effectiveness of EMFEM which was defined as a feedback message 
combining motivational and emotional content strategies. Results showed that (i) students’ motivation was 
increased; (ii) students’ positive attitudes toward IT increased; (iii) students liked the EMFEM and the 
instructor’s teaching style; (iv) students had a closer and friendlier relationship with the instructor; (v) by the end 
of the course, students were satisfied with the course; (vi) students started to use more emotional content; (vii) 
students enjoyed having personalized EMFEM and requested to have more EMFEM; and (ix) students had 
positive experiences by the end of the course. 

Online instructors should be aware of their students’ needs, perhaps by answering a few questions such as: what, 
when and how the students would like to learn. The other important question is to ask what kind of instructor 
students want. This question may be especially important for online instructors. They should ask themselves 
—what kind of instructor am I? In this study, the instructor chose to be friendly, social, and responsive. The 
results of this study should be considered in this light. Students’ responses, their interaction with the instructor 
and their feelings toward the instructor were related to instructors’ teaching skills, personality and educational 
philosophy. 

In future studies, researchers should be aware of the relatedness of online instructors’ decisions about their 
teaching roles in online learning environments. This study, despite its limitations, provided an assessment of a 
new kind of feedback message for online learning environments. EMFEM was supported by emotional content 
and motivational strategies for encouraging students during the course. Further research might clarify how 
EMFEM works in online learning environments and explore strategies for making instructor use of EMFEM 
more efficient. 
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